We revisit the domain of off-policy policy optimization in RL from the perspective of coordinate ascent. One commonly-used approach is to leverage the off-policy policy gradient to optimize a surrogate objective -- the total discounted in expectation return of the target policy with respect to the state distribution of the behavior policy. However, this approach has been shown to suffer from the distribution mismatch issue, and therefore significant efforts are needed for correcting this mismatch either via state distribution correction or a counterfactual method. In this paper, we rethink off-policy learning via Coordinate Ascent Policy Optimization (CAPO), an off-policy actor-critic algorithm that decouples policy improvement from the state distribution of the behavior policy without using the policy gradient. This design obviates the need for distribution correction or importance sampling in the policy improvement step of off-policy policy gradient. We establish the global convergence of CAPO with general coordinate selection and then further quantify the convergence rates of several instances of CAPO with popular coordinate selection rules, including the cyclic and the randomized variants of CAPO. We then extend CAPO to neural policies for a more practical implementation. Through experiments, we demonstrate that CAPO provides a competitive approach to RL in practice.

本文提出了将语音分离和增强（SSE）集成到ESPNET工具包中的最新进展。与以前的ESPNET-SE工作相比，已经添加了许多功能，包括最近的最新语音增强模型，并具有各自的培训和评估食谱。重要的是，已经设计了一个新界面，以灵活地将语音增强前端与其他任务相结合，包括自动语音识别（ASR），语音翻译（ST）和口语理解（SLU）。为了展示这种集成，我们在精心设计的合成数据集上进行了实验，用于嘈杂的多通道ST和SLU任务，可以用作未来研究的基准语料库。除了这些新任务外，我们还使用Chime-4和WSJ0-2MIX进行基准多链和单渠道SE方法。结果表明，即使在ASR以外的任务，尤其是在多频道方案中，SE前端与后端任务的集成也是一个有希望的研究方向。该代码可在https://github.com/espnet/espnet上在线获得。 HuggingFace上发布了这项工作的另一个贡献的多通道ST和SLU数据集。

基于深度的学习模型具有显着提高了语音分离的性能与鸡尾酒会等输入混合物。突出的方法（例如，频域和时域语音分离）通常建立回归模型以使用基于掩蔽的设计和信号级损耗标准（例如，MSE或SI-）来预测来自混合的地面真理语音。（例如，MSE或SI- SNR）。这项研究首次证明了基于综合的方法也可以在这个问题上表现良好，具有很大的灵活性和强大的潜力。具体地，我们提出了一种基于离散符号的识别的新型语音分离/增强模型，并将语音分离/增强相关任务的范例转换为分类。通过利用具有离散符号的输入的合成模型，在预测离散符号序列之后，可以重新合成每个目标语音。基于WSJ0-2MIX和VCTK-Noisy Corpora的评估结果在各种设置中表明，我们所提出的方法可以稳定地用高语音质量且没有任何干扰的分离的语音，这难以避免基于回归的方法。此外，通过忽略的聆听质量损失，通过我们的方法可以轻松实现增强/分离演讲的扬声器转换。

扩散概率模型已经证明了通过配对的扩散和反向过程模拟自然图像和原始音频波形的出色能力。可以利用反向过程的唯一特性（即，从高斯噪声和噪声信号中消除非目标信号）来恢复清洁信号。基于此属性，我们提出了一种基于扩散的基于概率模型的语言增强（漫反射）模型，其旨在从嘈杂的信号中恢复清洁语音信号。所提出的漫射模型的基本架构类似于差异 - 一种具有相对低的计算成本和足迹的高质量音频波形生成模型。为了获得更好的增强性能，我们设计了先进的反向过程，称为支持性反向过程，在每个时间步骤到预测的语音，这会增加噪音。实验结果表明，漫反射率与标准化语音银行语料库SE任务上的相关音频生成模型相当的性能。此外，相对于普遍建议的完整采样时间表，所提出的支持逆过程特别改善了快速采样，采取了几个步骤，从而产生更好的增强，从而通过传统的完整步长推断过程。

以前的研究已经证实了利用明晰度信息达到改善的语音增强（SE）性能的有效性。通过使用铰接特征的地点/方式增强原始声学特征，可以引导SE过程考虑执行增强时输入语音的剖视特性。因此，我们认为关节属性的上下文信息应包括有用的信息，并可以进一步利用不同的语言。在这项研究中，我们提出了一个SE系统，通过优化英语和普通话的增强演讲中的上下文清晰度信息来提高其性能。我们通过联合列车与端到端的自动语音识别（E2E ASR）模型进行联合列车，预测广播序列（BPC）而不是单词序列的序列。同时，开发了两种培训策略，以基于基于BPC的ASR：多任务学习和深度特征培训策略来培训SE系统。 Timit和TMhint DataSet上的实验结果证实了上下文化学信息促进了SE系统，以实现比传统声学模型（AM）更好的结果。此外，与用单声道ASR培训的另一SE系统相比，基于BPC的ASR（提供上下文化学信息）可以在不同的信噪比（SNR）下更有效地改善SE性能。

Deep learning models can achieve high accuracy when trained on large amounts of labeled data. However, real-world scenarios often involve several challenges: Training data may become available in installments, may originate from multiple different domains, and may not contain labels for training. Certain settings, for instance medical applications, often involve further restrictions that prohibit retention of previously seen data due to privacy regulations. In this work, to address such challenges, we study unsupervised segmentation in continual learning scenarios that involve domain shift. To that end, we introduce GarDA (Generative Appearance Replay for continual Domain Adaptation), a generative-replay based approach that can adapt a segmentation model sequentially to new domains with unlabeled data. In contrast to single-step unsupervised domain adaptation (UDA), continual adaptation to a sequence of domains enables leveraging and consolidation of information from multiple domains. Unlike previous approaches in incremental UDA, our method does not require access to previously seen data, making it applicable in many practical scenarios. We evaluate GarDA on two datasets with different organs and modalities, where it substantially outperforms existing techniques.

The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.

As one of the prevalent methods to achieve automation systems, Imitation Learning (IL) presents a promising performance in a wide range of domains. However, despite the considerable improvement in policy performance, the corresponding research on the explainability of IL models is still limited. Inspired by the recent approaches in explainable artificial intelligence methods, we proposed a model-agnostic explaining framework for IL models called R2RISE. R2RISE aims to explain the overall policy performance with respect to the frames in demonstrations. It iteratively retrains the black-box IL model from the randomized masked demonstrations and uses the conventional evaluation outcome environment returns as the coefficient to build an importance map. We also conducted experiments to investigate three major questions concerning frames' importance equality, the effectiveness of the importance map, and connections between importance maps from different IL models. The result shows that R2RISE successfully distinguishes important frames from the demonstrations.

Compressed videos often exhibit visually annoying artifacts, known as Perceivable Encoding Artifacts (PEAs), which dramatically degrade video visual quality. Subjective and objective measures capable of identifying and quantifying various types of PEAs are critical in improving visual quality. In this paper, we investigate the influence of four spatial PEAs (i.e. blurring, blocking, bleeding, and ringing) and two temporal PEAs (i.e. flickering and floating) on video quality. For spatial artifacts, we propose a visual saliency model with a low computational cost and higher consistency with human visual perception. In terms of temporal artifacts, self-attention based TimeSFormer is improved to detect temporal artifacts. Based on the six types of PEAs, a quality metric called Saliency-Aware Spatio-Temporal Artifacts Measurement (SSTAM) is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art metrics. We believe that SSTAM will be beneficial for optimizing video coding techniques.

We propose a distributionally robust return-risk model for Markov decision processes (MDPs) under risk and reward ambiguity. The proposed model optimizes the weighted average of mean and percentile performances, and it covers the distributionally robust MDPs and the distributionally robust chance-constrained MDPs (both under reward ambiguity) as special cases. By considering that the unknown reward distribution lies in a Wasserstein ambiguity set, we derive the tractable reformulation for our model. In particular, we show that that the return-risk model can also account for risk from uncertain transition kernel when one only seeks deterministic policies, and that a distributionally robust MDP under the percentile criterion can be reformulated as its nominal counterpart at an adjusted risk level. A scalable first-order algorithm is designed to solve large-scale problems, and we demonstrate the advantages of our proposed model and algorithm through numerical experiments.